Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision Supplementary Materials
نویسندگان
چکیده
Similar to the spatial transformer network introduced in [1], we propose a 2-step procedure: (1) performing dense sampling from input volume (in 3D world coordinates) to output volume (in screen coordinates), and (2) flattening the 3D spatial output across disparity dimension. In the experiment, we assume that transformation matrix is always given as input, parametrized by the viewpoint α. Again, the 3D point (xi , y s i , z s i ) in input volume V ∈ RH×W×D and corresponding point (xi, y i , di) in output volume U ∈ RH×W ′×D′ is linked by perspective transformation matrix Θ4×4. Here, (W,H,D) and (W ′, H ′, D′) are the width, height and depth of input and output volume, respectively. x s i y i z i 1 = θ11 θ12 θ13 θ14 θ21 θ22 θ23 θ24 θ31 θ32 θ33 θ34 θ41 θ42 θ43 θ44 x̃i t ỹi t z̃i t 1 (2)
منابع مشابه
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Understanding the 3D world is a fundamental problem in computer vision. However, learning a good representation of 3D objects is still an open problem due to the high dimensionality of the data and many factors of variation involved. In this work, we investigate the task of single-view 3D object reconstruction from a learning agent’s perspective. We formulate the learning process as an interact...
متن کاملWeakly Supervised Generative Adversarial Networks for 3D Reconstruction
Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enab...
متن کاملWeakly supervised 3D Reconstruction with Adversarial Constraint
Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enab...
متن کاملUnsupervised learning through one-shot image-based shape reconstruction
Objects are three-dimensional entities, but visual observations are largely 2D. Inferring 3D properties from individual 2D views is thus a generically useful skill that is critical to object perception. We ask the question: can we learn useful image representations by explicitly training a system to infer 3D shape from 2D views? The few prior attempts at single view 3D reconstruction all target...
متن کاملLearning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
We study the problem of 3D object generation. We propose a novel framework, namely 3D Generative Adversarial Network (3D-GAN), which generates 3D objects from a probabilistic space by leveraging recent advances in volumetric convolutional networks and generative adversarial nets. The benefits of our model are three-fold: first, the use of an adversarial criterion, instead of traditional heurist...
متن کامل